Papers with objective function
Non-Autoregressive Sequence Generation (2022.acl-tutorials)
Copied to clipboard
| Challenge: | Non-autoregressive sequence generation (NAR) models generate output sequences in parallel to speed up generation process. |
| Approach: | This tutorial provides a thorough introduction and review of non-autoregressive sequence generation . it aims to generate the entire or partial output sequences in parallel to speed up the generation process . |
| Outcome: | This tutorial provides a thorough introduction and review of non-autoregressive sequence generation . it aims to reduce the performance gap between state-of-the-art models due to lack of modeling power . |
Training with Adversaries to Improve Faithfulness of Attention in Neural Machine Translation (2020.aacl-srw)
Copied to clipboard
| Challenge: | Existing approaches to measure faithfulness of neural machine translation models are based on stress tests and a novel objective that rewards faithful behaviour by the model through probability divergence. |
| Approach: | They propose a measure of faithfulness for neural machine translation models based on stress tests and measuring faithfulness based upon how often the model output changes. |
| Outcome: | The proposed objective increases faithfulness without reducing translation quality and can even improve translation quality in some cases. |
Unsupervised Paraphrasing by Simulated Annealing (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to generate accurate and different-appearing paraphrases require massive parallel samples for training. |
| Approach: | They propose a novel approach that accomplishes Unsupervised Paraphrasing by Simulated Annealing by performing local editing. |
| Outcome: | The proposed approach outperforms existing models in automatic and human evaluations on Quora, Wikianswers, MSCOCO, and Twitter. |
SYSTRAN @ WNGT 2019: DGT Task (D19-56)
Copied to clipboard
| Challenge: | SYSTRAN participates in Document-level generation and trans-lation (DGT) task . data-to-text generation tasks are difficult because of the content selection and text generation data. |
| Approach: | They propose a Transformer-based datato-text generation model which jointly learns content selection and text generation. |
| Outcome: | The proposed model outperforms current state-of-the-art system on BLEU, content selection precision and content ordering metics. |
A Variational Approach to Weakly Supervised Document-Level Multi-Aspect Sentiment Classification (N19-1)
Copied to clipboard
| Challenge: | Existing weakly supervised methods for document-level multi-aspect sentiment classification are not easy to obtain. |
| Approach: | They propose a variational approach to weakly supervised document-level multi-aspect sentiment classification using target-opinion word pairs as "supervision" they aim to learn a sentiment polarity classifier by optimizing the lower bound . |
| Outcome: | The proposed method outperforms weakly supervised baselines on TripAdvisor and BeerAdvocate datasets and can be comparable to state-of-the-art supervised methods with hundreds of labels per aspect. |
Nested Named Entity Recognition via Second-best Sequence Learning and Decoding (2020.tacl-1)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is the task of identifying text spans associated with proper names and classifying them according to their semantic class such as person or organization. |
| Approach: | They propose a method that treats the tag sequence for nested entities as the second best path within the span of their parent entity. |
| Outcome: | The proposed method achieves F1-scores of 85.82%, 84.34%, and 77.36% on ACE-2004, ACE 2005, and GENIA datasets. |
TrojFSP: Trojan Insertion in Few-shot Prompt Tuning (2024.naacl-long)
Copied to clipboard
| Challenge: | Prompt tuning on a few data samples presents security issues, e.g., Trojan attacks. |
| Approach: | They propose a method to transfer established data poisoning attacks directly to few-shot prompt tuning, a technique to address the poisoned imbalance issue. |
| Outcome: | The proposed method achieves an ASR of over 99% while maintaining negligible decreases in CDA. |
An Uncertainty-Aware Encoder for Aspect Detection (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for aspect detection use seed words as priors or features of topic models. |
| Approach: | They propose a weakly-supervised method to exploit seed words for aspect detection . goal is approximating similarity between segments and aspects and ground-truth similarity generated from seed words. |
| Outcome: | The proposed method outperforms previous work on several benchmarks in various domains. |
SHARP: Search-Based Adversarial Attack for Structured Prediction (2022.findings-naacl)
Copied to clipboard
| Challenge: | SHARP is a new attack method for structured prediction models that solves several challenges. |
| Approach: | They propose a black-box adversarial attack method that uses a search-based optimization problem to attack adversarials. |
| Outcome: | The proposed method performs more potent attack than pioneer arts on two structured prediction tasks. |
Simple and Effective Multi-Paragraph Reading Comprehension (P18-1)
Copied to clipboard
| Challenge: | Existing question answering models cannot scale beyond short paragraphs, so adapting a model to document-level input is difficult. |
| Approach: | They propose a method of adapting neural paragraph-level question answering models to document input. |
| Outcome: | The proposed method achieves state-of-the-art on TriviaQA and SQuAD and a 10 point gain on SQuADA. |
SwarmAgentic: Towards Fully Automated Agentic System Generation via Swarm Intelligence (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing agentic system generation frameworks lack autonomy, autonomy, and functionality . current frameworks are too rigid, limiting adaptability and scalability. |
| Approach: | They propose a framework that fully automates agentic system generation, optimization, and collaboration . they construct agents from scratch and jointly refine functionality and coordination . |
| Outcome: | The proposed framework outperforms ADAS on six real-world, open-ended, and exploratory tasks on the TravelPlanner benchmark. |
BEAR: A Unified Framework for Evaluating Relational Knowledge in Causal and Masked Language Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods to evaluate LMs rely on objective function and are therefore limited to masked or causal LM types. |
| Approach: | They propose an approach that uses an LM’s inherent ability to estimate the log-likelihood of any given textual statement. |
| Outcome: | The proposed framework can probe for knowledge across different LM types. |
MeMoTune: A Measure and Moment-Driven Fine-Tuning Framework for Quantized Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods combine quantization with parameter-efficient fine-tuning but fail to meet practical performance requirements. |
| Approach: | They propose a measure and moment approach to optimize objective function for superior fine-tuning results by scaling the update process through a gradient. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on tasks like text generation, summarization, and understanding. |
A Generic Method for Fine-grained Category Discovery in Natural Language Texts (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for fine-grained category discovery neglect semantic similarities of fine-grain categories. |
| Approach: | They propose a method that detects fine-grained clusters of semantically similar texts guided by a novel objective function. |
| Outcome: | The proposed method surpasses state-of-the-art methods on three benchmark tasks. |
Bidirectional Transformer Reranker for Grammatical Error Correction (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained seq2seq models suffer from a prediction bias due to their unidirectional decoding. |
| Approach: | They propose a bidirectional Transformer reranker that re-estimates the probability of each candidate sentence generated by pre-trained seq2seq models. |
| Outcome: | The proposed model improves on the original model and gives a 59.52 GLEU score on the JFLEG corpus. |
RobustSentEmbed: Robust Sentence Embeddings Using Adversarial Self-Supervised Contrastive Learning (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing pre-trained language models exhibit poor generalization and robustness in adversarial settings. |
| Approach: | They propose a self-supervised sentence embedding framework that improves generalization and robustness against adversarial attacks. |
| Outcome: | The proposed framework reduces the success rate of adversarial attacks by almost half . it also improves semantic text similarity tasks and various transfer tasks . |
Measuring and Improving Faithfulness of Attention in Neural Machine Translation (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing evidence for faithfulness of neural machine translation models is lacking. |
| Approach: | They propose a novel objective that rewards faithful behaviour by the model through probability divergence and a differentiable objective that can increase faithfulness without reducing the translation quality. |
| Outcome: | The proposed objective increases faithfulness without reducing translation quality and can even improve translation quality in some cases. |
ZmBART: An Unsupervised Cross-lingual Transfer Framework for Language Generation (2021.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in NLP focus on large annotated training data. |
| Approach: | They propose an unsupervised framework that does not use parallel or pseudo-parallel/back-translated data. |
| Outcome: | The proposed framework does not use parallel or pseudo-parallel/back-translated data. |
A Regularization Approach for Incorporating Event Knowledge and Coreference Relations into Neural Discourse Parsing (D19-1)
Copied to clipboard
| Challenge: | Existing approaches to discourse parsing use commonsense knowledge and linguistic constraints to integrate them into neural network models. |
| Approach: | They propose a knowledge regularization approach that integrates linguistic constraints with contexts for deriving word representations. |
| Outcome: | The proposed approach outperforms previous systems on the benchmark dataset PDTB for discourse parsing. |
Towards Less Generic Responses in Neural Conversation Models: A Statistical Re-weighting Method (D18-1)
Copied to clipboard
| Challenge: | Experimental results show that Sequence-to-sequence models tend to generate generic/dull responses . |
| Approach: | They propose a statistical re-weighting method that assigns different weights for multiple responses of the same query. |
| Outcome: | The proposed method improves acceptance rate of generated responses and significantly reduces generated generic responses. |
Stronger Universal and Transferable Attacks by Suppressing Refusals (2025.naacl-long)
Copied to clipboard
| Challenge: | Efforts have focused on aligning models to human preferences (RLHF) . yet, it is believed that such optimization-based attacks are sample-specific. |
| Approach: | They propose an algorithm to embed a "safety feature" into models to make them safe for mass deployment. |
| Outcome: | The proposed attack achieves 25% success rate against the state-of-the-art Circuit Breaker defense, compared to 2.5% by white-box GCG. |
Cross-Lingual Document Retrieval with Smooth Learning (2020.coling-main)
Copied to clipboard
| Challenge: | Cross-lingual document search is an information retrieval task in which the queries’ language and the documents’ language are different. |
| Approach: | They propose a robust framework that measures the relevance and a loss function that is a novel objective function. |
| Outcome: | The proposed framework achieves significant gains under commonly used ranking metrics on cross-lingual document retrieval task in a variety of languages. |
Toward Interactive Regional Understanding in Vision-Large Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing image-text pairs capture only coarse and global information of an image, leading to a limitation in their regional understanding ability. |
| Approach: | They propose a model with explicit regional modeling capabilities that allows VLP models to understand user-indicated image regions. |
| Outcome: | The proposed model performs better on zero-shot region understanding tasks without compromising its ability for global image understanding. |
Experimenting with Power Divergences for Language Modeling (D19-1)
Copied to clipboard
| Challenge: | Language models are an important component in many NLP tasks, where they provide prior knowledge on the language used. |
| Approach: | They propose to use power divergences to prioritize learning on frequent or rare words . they use a sample-based objective to approximate a softmax and noise-constrained estimate . |
| Outcome: | The proposed power divergences can be used to prioritize learning on the frequent or rare words and lead to general performance improvements. |
A Deep Neural Information Fusion Architecture for Textual Network Embeddings (D19-1)
Copied to clipboard
| Challenge: | Textual network embeddings aim to learn a low-dimensional representation for every node in the network while seeking to retain the original network information. |
| Approach: | They propose a deep neural architecture to fuse the two kinds of informations into one representation. |
| Outcome: | The proposed model outperforms the comparing methods on all three datasets. |
Sequence-to-Nuggets: Nested Entity Mention Detection via Anchor-Region Networks (P19-1)
Copied to clipboard
| Challenge: | Named entity recognition (NER) approaches restrict each word belonging to at most one entity mention. |
| Approach: | They propose to model and leverage the head-driven phrase structures of entity mentions to solve this problem. |
| Outcome: | The proposed architecture achieves state-of-the-art on three standard nested entity mention detection benchmarks. |
Incorporating Priors with Feature Attribution on Text Classification (P19-1)
Copied to clipboard
| Challenge: | Feature attribution methods are used to help users interpret complex models. |
| Approach: | They propose a feature attribution method that integrates feature attributed features into the objective function to allow machine learning practitioners to incorporate priors in model building. |
| Outcome: | The proposed method reduces undesired model biases without a tradeoff on the original task and improves classifier performance in scarce data setting. |
RadialRouter: Structured Representation for Efficient and Robust Large Language Models Routing (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Current routing methods are limited in exploring the connection between query and LLM characteristics. |
| Approach: | They propose a framework for LLM routing that uses a transformer-based backbone and a radial structure to articulate the query-LLMs relationship. |
| Outcome: | The proposed framework outperforms existing routing methods by 9.2% and 5.8% on RouterBench. |
Searching for the Most Human-like Emergent Language (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on emergent communication systems to generate languages with high statistical similarity to human languages has not been done. |
| Approach: | They propose to optimize a signalling game-based emergent communication environment to generate state-of-the-art emergentic languages with a high degree of similarity to human language. |
| Outcome: | The proposed language generates state-of-the-art on XferBench benchmark, demonstrating its similarity to human language and entropy-minimization properties. |